{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example Model Servers with Seldon"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequisites\n",
    "You will need\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - A running Kubernetes cluster with kubectl authenticated\n",
    " - [seldon-core Python package](https://pypi.org/project/seldon-core/) (```pip install seldon-core>=0.2.6.1```)\n",
    " - [Helm client](https://helm.sh/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating a Kubernetes Cluster\n",
    "\n",
    "Follow the [Kubernetes documentation to create a cluster](https://kubernetes.io/docs/setup/).\n",
    "\n",
    "Once created ensure ```kubectl``` is authenticated against the running cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace/seldon created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Context \"minikube\" modified.\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl config set-context $(kubectl config current-context) --namespace=seldon"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "clusterrolebinding.rbac.authorization.k8s.io/kube-system-cluster-admin created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Install Helm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount/tiller created\n",
      "clusterrolebinding.rbac.authorization.k8s.io/tiller created\n",
      "$HELM_HOME has been configured at /home/clive/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"tiller-deploy\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"tiller-deploy\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/tiller-deploy -n kube-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start seldon-core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Sat Aug 31 10:58:33 2019\n",
      "NAMESPACE: seldon-system\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ClusterRole\n",
      "NAME                          AGE\n",
      "seldon-operator-manager-role  1s\n",
      "\n",
      "==> v1/ClusterRoleBinding\n",
      "NAME                                 AGE\n",
      "seldon-operator-manager-rolebinding  1s\n",
      "\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-config            1     1s\n",
      "seldon-spartakus-config  1     1s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                         READY  STATUS             RESTARTS  AGE\n",
      "seldon-operator-controller-manager-0         0/1    ContainerCreating  0         1s\n",
      "seldon-spartakus-volunteer-7d6dd98f89-2jgbp  0/1    ContainerCreating  0         1s\n",
      "\n",
      "==> v1/Secret\n",
      "NAME                                   TYPE    DATA  AGE\n",
      "seldon-operator-webhook-server-secret  Opaque  0     1s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                                        TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)  AGE\n",
      "seldon-operator-controller-manager-service  ClusterIP  10.106.64.198  <none>       443/TCP  1s\n",
      "webhook-server-service                      ClusterIP  10.97.147.240  <none>       443/TCP  1s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                              SECRETS  AGE\n",
      "seldon-core-seldon-core-operator  1        1s\n",
      "seldon-spartakus-volunteer        1        1s\n",
      "\n",
      "==> v1/StatefulSet\n",
      "NAME                                READY  AGE\n",
      "seldon-operator-controller-manager  0/1    1s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  1s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  1s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        READY  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  0/1    1           0          1s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core-operator --name seldon-core  --set image.pullPolicy=IfNotPresent --set usageMetrics.enabled=true   --namespace seldon-system"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "partitioned roll out complete: 1 new pods have been updated...\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/seldon-controller-manager -n seldon-system"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup Ingress\n",
    "Please note: There are reported gRPC issues with ambassador (see https://github.com/SeldonIO/seldon-core/issues/473)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   ambassador\n",
      "LAST DEPLOYED: Sat Aug 31 10:56:45 2019\n",
      "NAMESPACE: seldon\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/Deployment\n",
      "NAME        READY  UP-TO-DATE  AVAILABLE  AGE\n",
      "ambassador  0/3    3           0          1s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                        READY  STATUS             RESTARTS  AGE\n",
      "ambassador-5f7fd859b-4bhj7  0/1    ContainerCreating  0         0s\n",
      "ambassador-5f7fd859b-6br5r  0/1    ContainerCreating  0         0s\n",
      "ambassador-5f7fd859b-rr6pj  0/1    ContainerCreating  0         0s\n",
      "\n",
      "==> v1/Service\n",
      "NAME              TYPE          CLUSTER-IP      EXTERNAL-IP  PORT(S)                     AGE\n",
      "ambassador        LoadBalancer  10.111.228.245  <pending>    80:31947/TCP,443:32642/TCP  1s\n",
      "ambassador-admin  ClusterIP     10.99.206.49    <none>       8877/TCP                    1s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME        SECRETS  AGE\n",
      "ambassador  1        1s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME             AGE\n",
      "ambassador       1s\n",
      "ambassador-crds  1s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME             AGE\n",
      "ambassador       1s\n",
      "ambassador-crds  1s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                          AGE\n",
      "authservices.getambassador.io                 1s\n",
      "consulresolvers.getambassador.io              1s\n",
      "kubernetesendpointresolvers.getambassador.io  1s\n",
      "kubernetesserviceresolvers.getambassador.io   1s\n",
      "mappings.getambassador.io                     1s\n",
      "modules.getambassador.io                      1s\n",
      "ratelimitservices.getambassador.io            1s\n",
      "tcpmappings.getambassador.io                  1s\n",
      "tlscontexts.getambassador.io                  1s\n",
      "tracingservices.getambassador.io              1s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Congratuations! You've successfully installed Ambassador.\n",
      "\n",
      "For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.\n",
      "\n",
      "To get the IP address of Ambassador, run the following commands:\n",
      "NOTE: It may take a few minutes for the LoadBalancer IP to be available.\n",
      "     You can watch the status of by running 'kubectl get svc -w  --namespace seldon ambassador'\n",
      "\n",
      "  On GKE/Azure:\n",
      "  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')\n",
      "\n",
      "  On AWS:\n",
      "  export SERVICE_IP=$(kubectl get svc --namespace seldon ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')\n",
      "\n",
      "  echo http://$SERVICE_IP:\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install stable/ambassador --name ambassador --set crds.keep=false"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"ambassador\" rollout to finish: 0 of 3 updated replicas are available...\n",
      "Waiting for deployment \"ambassador\" rollout to finish: 1 of 3 updated replicas are available...\n",
      "Waiting for deployment \"ambassador\" rollout to finish: 2 of 3 updated replicas are available...\n",
      "deployment \"ambassador\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deployment.apps/ambassador"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Port Forward to Ambassador\n",
    "\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n seldon -l app.kubernetes.io/name=ambassador -o jsonpath='{.items[0].metadata.name}') -n seldon 8003:8080\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve SKlearn Iris Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34;01mapiVersion\u001b[39;49;00m: machinelearning.seldon.io/v1alpha2\r\n",
      "\u001b[34;01mkind\u001b[39;49;00m: SeldonDeployment\r\n",
      "\u001b[34;01mmetadata\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: sklearn\r\n",
      "\u001b[34;01mspec\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: iris\r\n",
      "  \u001b[34;01mpredictors\u001b[39;49;00m:\r\n",
      "  - \u001b[34;01mgraph\u001b[39;49;00m:\r\n",
      "      \u001b[34;01mchildren\u001b[39;49;00m: []\r\n",
      "      \u001b[34;01mimplementation\u001b[39;49;00m: SKLEARN_SERVER\r\n",
      "      \u001b[34;01mmodelUri\u001b[39;49;00m: gs://seldon-models/sklearn/iris\r\n",
      "      \u001b[34;01mname\u001b[39;49;00m: classifier\r\n",
      "    \u001b[34;01mname\u001b[39;49;00m: default\r\n",
      "    \u001b[34;01mreplicas\u001b[39;49;00m: 1\r\n",
      "    \u001b[34;01msvcOrchSpec\u001b[39;49;00m: \r\n",
      "      \u001b[34;01menv\u001b[39;49;00m: \r\n",
      "      - \u001b[34;01mname\u001b[39;49;00m: SELDON_LOG_LEVEL\r\n",
      "        \u001b[34;01mvalue\u001b[39;49;00m: DEBUG\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize ../servers/sklearnserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/sklearn created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/sklearnserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"iris-default-4903e3c\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"iris-default-4903e3c\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/iris-default-4903e3c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"sklearn\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 4\n",
      "    values: 0.44028212923599264\n",
      "    values: 0.22694244373903638\n",
      "    values: 0.08693601526817618\n",
      "    values: 0.721446205469061\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "meta {\n",
      "  puid: \"2k933j6cl1sq2kgh24ck96fmi8\"\n",
      "  requestPath {\n",
      "    key: \"classifier\"\n",
      "    value: \"seldonio/sklearnserver_rest:0.2\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"t:0\"\n",
      "  names: \"t:1\"\n",
      "  names: \"t:2\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 3\n",
      "    values: 0.3328333104192785\n",
      "    values: 0.352243232066047\n",
      "    values: 0.3149234575146744\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,4))\n",
    "print(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"sklearn\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/sklearnserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve XGBoost Iris Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34;01mapiVersion\u001b[39;49;00m: machinelearning.seldon.io/v1alpha2\r\n",
      "\u001b[34;01mkind\u001b[39;49;00m: SeldonDeployment\r\n",
      "\u001b[34;01mmetadata\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: xgboost\r\n",
      "\u001b[34;01mspec\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: iris\r\n",
      "  \u001b[34;01mpredictors\u001b[39;49;00m:\r\n",
      "  - \u001b[34;01mgraph\u001b[39;49;00m:\r\n",
      "      \u001b[34;01mchildren\u001b[39;49;00m: []\r\n",
      "      \u001b[34;01mimplementation\u001b[39;49;00m: XGBOOST_SERVER\r\n",
      "      \u001b[34;01mmodelUri\u001b[39;49;00m: gs://seldon-models/xgboost/iris\r\n",
      "      \u001b[34;01mname\u001b[39;49;00m: classifier\r\n",
      "    \u001b[34;01mname\u001b[39;49;00m: default\r\n",
      "    \u001b[34;01mreplicas\u001b[39;49;00m: 1\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize ../servers/xgboostserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/xgboost created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/xgboostserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"iris-default-af1783b\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"iris-default-af1783b\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/iris-default-af1783b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"xgboost\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 4\n",
      "    values: 0.2379841226697781\n",
      "    values: 0.6766416345988621\n",
      "    values: 0.8055747864263723\n",
      "    values: 0.020395604144540558\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "meta {\n",
      "  puid: \"gku8gaufrl69ra8btc1n1dih97\"\n",
      "  requestPath {\n",
      "    key: \"classifier\"\n",
      "    value: \"seldonio/xgboostserver_rest:0.2\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    values: 0.0\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,4))\n",
    "print(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"xgboost\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/xgboostserver/samples/iris.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve Tensorflow MNIST Model\n",
    "\n",
    "**Will only work on a GCP Kubernetes Cluster**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34;01mapiVersion\u001b[39;49;00m: machinelearning.seldon.io/v1alpha2\r\n",
      "\u001b[34;01mkind\u001b[39;49;00m: SeldonDeployment\r\n",
      "\u001b[34;01mmetadata\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: tfserving\r\n",
      "\u001b[34;01mspec\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: mnist\r\n",
      "  \u001b[34;01mpredictors\u001b[39;49;00m:\r\n",
      "  - \u001b[34;01mgraph\u001b[39;49;00m:\r\n",
      "      \u001b[34;01mchildren\u001b[39;49;00m: []\r\n",
      "      \u001b[34;01mimplementation\u001b[39;49;00m: TENSORFLOW_SERVER\r\n",
      "      \u001b[34;01mmodelUri\u001b[39;49;00m: gs://seldon-models/tfserving/mnist-model\r\n",
      "      \u001b[34;01mname\u001b[39;49;00m: mnist-model\r\n",
      "      \u001b[34;01mparameters\u001b[39;49;00m:\r\n",
      "        - \u001b[34;01mname\u001b[39;49;00m: signature_name\r\n",
      "          \u001b[34;01mtype\u001b[39;49;00m: STRING\r\n",
      "          \u001b[34;01mvalue\u001b[39;49;00m: predict_images\r\n",
      "        - \u001b[34;01mname\u001b[39;49;00m: model_name\r\n",
      "          \u001b[34;01mtype\u001b[39;49;00m: STRING\r\n",
      "          \u001b[34;01mvalue\u001b[39;49;00m: mnist-model\r\n",
      "    \u001b[34;01mname\u001b[39;49;00m: default\r\n",
      "    \u001b[34;01mreplicas\u001b[39;49;00m: 1\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize ../servers/tfserving/samples/mnist_rest.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/tfserving created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/tfserving/samples/mnist_rest.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Waiting for deployment \"mnist-default-4f27cf3\" rollout to finish: 0 of 1 updated replicas are available...\n",
      "deployment \"mnist-default-4f27cf3\" successfully rolled out\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/mnist-default-4f27cf3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"tfserving\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 784\n",
      "    values: 0.6960851737435524\n",
      "    values: 0.31659351175832906\n",
      "    values: 0.8839475487253241\n",
      "    values: 0.3088659639767263\n",
      "    values: 0.8891109022304765\n",
      "    values: 0.19015731671987868\n",
      "    values: 0.2084631221539136\n",
      "    values: 0.4544208075311512\n",
      "    values: 0.8926698161003384\n",
      "    values: 0.19433553241548673\n",
      "    values: 0.13531711500693755\n",
      "    values: 0.8101485252410934\n",
      "    values: 0.357998040732349\n",
      "    values: 0.8149058111283166\n",
      "    values: 0.7166138223407796\n",
      "    values: 0.5481713626892256\n",
      "    values: 0.8970834930204503\n",
      "    values: 0.8913760644394269\n",
      "    values: 0.8865946422300978\n",
      "    values: 0.7283231572016693\n",
      "    values: 0.061513546769038197\n",
      "    values: 0.8946366478131781\n",
      "    values: 0.08234186387502296\n",
      "    values: 0.10724101325268287\n",
      "    values: 0.6018581181011512\n",
      "    values: 0.2833759431809676\n",
      "    values: 0.12643611214180328\n",
      "    values: 0.622302479013179\n",
      "    values: 0.995550102668352\n",
      "    values: 0.3328497257873455\n",
      "    values: 0.9302707947391331\n",
      "    values: 0.4468558590780892\n",
      "    values: 0.42920347680284276\n",
      "    values: 0.5372591933577155\n",
      "    values: 0.11270788071998872\n",
      "    values: 0.28988672422908424\n",
      "    values: 0.7416611600623187\n",
      "    values: 0.9971065442301893\n",
      "    values: 0.6929695492570231\n",
      "    values: 0.7146022772722823\n",
      "    values: 0.5225420262232614\n",
      "    values: 0.2957470630732245\n",
      "    values: 0.20180834104752754\n",
      "    values: 0.027983327167379057\n",
      "    values: 0.743259268294824\n",
      "    values: 0.9380320728626145\n",
      "    values: 0.15101646676991454\n",
      "    values: 0.7409886299539135\n",
      "    values: 0.8708450759891614\n",
      "    values: 0.7398479612053028\n",
      "    values: 0.47169861563557147\n",
      "    values: 0.4758555045353844\n",
      "    values: 0.051715042211538886\n",
      "    values: 0.9305707812109276\n",
      "    values: 0.7432651725586414\n",
      "    values: 0.7451248626685253\n",
      "    values: 0.9306479902282564\n",
      "    values: 0.3702371078940443\n",
      "    values: 0.22222707494585447\n",
      "    values: 0.22510121015280415\n",
      "    values: 0.6595390828716817\n",
      "    values: 0.5263762785871035\n",
      "    values: 0.962362278625186\n",
      "    values: 0.6547519478334944\n",
      "    values: 0.3838371764345332\n",
      "    values: 0.021925712793501195\n",
      "    values: 0.959785838899824\n",
      "    values: 0.10138166375879198\n",
      "    values: 0.8402941786304847\n",
      "    values: 0.2617904990637009\n",
      "    values: 0.8431358767809322\n",
      "    values: 0.27435662005154604\n",
      "    values: 0.46969436581758883\n",
      "    values: 0.3298464049115838\n",
      "    values: 0.23876202039200844\n",
      "    values: 0.11630513443656354\n",
      "    values: 0.02497429318900901\n",
      "    values: 0.8851278231876147\n",
      "    values: 0.5898975059337347\n",
      "    values: 0.11786756715910929\n",
      "    values: 0.2519907309564371\n",
      "    values: 0.030904070969085207\n",
      "    values: 0.7663182302598766\n",
      "    values: 0.14698768033501763\n",
      "    values: 0.3078100599039062\n",
      "    values: 0.9122021858583312\n",
      "    values: 0.9296836721719605\n",
      "    values: 0.48520841913653834\n",
      "    values: 0.47214943726962544\n",
      "    values: 0.4697355700654081\n",
      "    values: 0.6681340824357441\n",
      "    values: 0.6226153217980376\n",
      "    values: 0.7233315833674575\n",
      "    values: 0.6801846802579725\n",
      "    values: 0.7001260131784245\n",
      "    values: 0.2433027123936522\n",
      "    values: 0.9020985629858237\n",
      "    values: 0.7951204462482829\n",
      "    values: 0.00478415566826762\n",
      "    values: 0.6933112743162382\n",
      "    values: 0.49543836729405366\n",
      "    values: 0.9370649280049107\n",
      "    values: 0.8254719360253051\n",
      "    values: 0.5216725429531935\n",
      "    values: 0.3805339266721025\n",
      "    values: 0.8063426251076562\n",
      "    values: 0.31565964381890144\n",
      "    values: 0.4462362590261284\n",
      "    values: 0.12009061557498446\n",
      "    values: 0.18647448952252488\n",
      "    values: 0.054650278429649646\n",
      "    values: 0.6960188942790909\n",
      "    values: 0.5806706486723264\n",
      "    values: 0.6363740900229701\n",
      "    values: 0.1724671777720067\n",
      "    values: 0.27789202161257087\n",
      "    values: 0.8103112708833594\n",
      "    values: 0.42002413498208346\n",
      "    values: 0.05460906955805889\n",
      "    values: 0.8055664468835065\n",
      "    values: 0.612239544710043\n",
      "    values: 0.48166416135191403\n",
      "    values: 0.5384873778692566\n",
      "    values: 0.858583195130647\n",
      "    values: 0.32710971467043704\n",
      "    values: 0.690602669025769\n",
      "    values: 0.9859575511612774\n",
      "    values: 0.2110689318148241\n",
      "    values: 0.4894298760979662\n",
      "    values: 0.9688163873790141\n",
      "    values: 0.8056032882153414\n",
      "    values: 0.6835773039412582\n",
      "    values: 0.26122322588598335\n",
      "    values: 0.8156334102575398\n",
      "    values: 0.40025400630980157\n",
      "    values: 0.11067678668736436\n",
      "    values: 0.9496285172478437\n",
      "    values: 0.4734446376613902\n",
      "    values: 0.37166554642438254\n",
      "    values: 0.9783823684061197\n",
      "    values: 0.46758659667842317\n",
      "    values: 0.8109768916933405\n",
      "    values: 0.055520558757641614\n",
      "    values: 0.4528453142024328\n",
      "    values: 0.42985929391454825\n",
      "    values: 0.42143614581918454\n",
      "    values: 0.40306290301342784\n",
      "    values: 0.46406332299978104\n",
      "    values: 0.28975939298090014\n",
      "    values: 0.28947524591540175\n",
      "    values: 0.18631033413027365\n",
      "    values: 0.304377522923453\n",
      "    values: 0.46175693066239365\n",
      "    values: 0.1150837510886189\n",
      "    values: 0.6812154917443837\n",
      "    values: 0.9614488756919379\n",
      "    values: 0.04913394446037633\n",
      "    values: 0.40737285679723745\n",
      "    values: 0.4329428459912802\n",
      "    values: 0.5037427062521643\n",
      "    values: 0.8123848185422287\n",
      "    values: 0.655191928369746\n",
      "    values: 0.04527689434634741\n",
      "    values: 0.0873815126142633\n",
      "    values: 0.5529769822794395\n",
      "    values: 0.1889822510783703\n",
      "    values: 0.22165216714263847\n",
      "    values: 0.7378659785922592\n",
      "    values: 0.12365069730517098\n",
      "    values: 0.14410012880222267\n",
      "    values: 0.314434996897037\n",
      "    values: 0.1703689839679695\n",
      "    values: 0.6690574474130547\n",
      "    values: 0.6664710564253505\n",
      "    values: 0.9339641442677029\n",
      "    values: 0.5409462387630555\n",
      "    values: 0.26074766339540667\n",
      "    values: 0.6957378315803447\n",
      "    values: 0.6263815242810925\n",
      "    values: 0.8000587612574093\n",
      "    values: 0.6933560258476635\n",
      "    values: 0.4657656293471626\n",
      "    values: 0.7285362738265765\n",
      "    values: 0.5888972328606239\n",
      "    values: 0.6337069487682453\n",
      "    values: 0.7749635685108407\n",
      "    values: 0.8332180653637755\n",
      "    values: 0.7220249677284571\n",
      "    values: 0.9450056846774213\n",
      "    values: 0.2949990524468443\n",
      "    values: 0.7812177080310112\n",
      "    values: 0.0007479935505044777\n",
      "    values: 0.6472500190114214\n",
      "    values: 0.05500446345677934\n",
      "    values: 0.9273297074667317\n",
      "    values: 0.29508984112297865\n",
      "    values: 0.18139798598121393\n",
      "    values: 0.043420646475409\n",
      "    values: 0.748747838187905\n",
      "    values: 0.9696210129999953\n",
      "    values: 0.4444161356554087\n",
      "    values: 0.34929215119538615\n",
      "    values: 0.11437128360879367\n",
      "    values: 0.40878396270888695\n",
      "    values: 0.7525849341373413\n",
      "    values: 0.6734271192794087\n",
      "    values: 0.21882426715297365\n",
      "    values: 0.1845465469769656\n",
      "    values: 0.7646381533622951\n",
      "    values: 0.928688307970282\n",
      "    values: 0.5835209639658983\n",
      "    values: 0.9395910841618587\n",
      "    values: 0.5430694448462892\n",
      "    values: 0.7316370109389899\n",
      "    values: 0.7881513920529627\n",
      "    values: 0.7760949104722156\n",
      "    values: 0.275134393397644\n",
      "    values: 0.7082774551494976\n",
      "    values: 0.8747442566482749\n",
      "    values: 0.5960786141658948\n",
      "    values: 0.34820133974955747\n",
      "    values: 0.17431717975043237\n",
      "    values: 0.9897531863569138\n",
      "    values: 0.4600232569300692\n",
      "    values: 0.23383447908889976\n",
      "    values: 0.4596112078591956\n",
      "    values: 0.8360406626702981\n",
      "    values: 0.7272870967626213\n",
      "    values: 0.7608953726381754\n",
      "    values: 0.6737369575965617\n",
      "    values: 0.7731532080628881\n",
      "    values: 0.348797025312249\n",
      "    values: 0.9790666072921824\n",
      "    values: 0.353185101375623\n",
      "    values: 0.4562595538413017\n",
      "    values: 0.6394333896676226\n",
      "    values: 0.3851883983757808\n",
      "    values: 0.751879328723079\n",
      "    values: 0.749543184886521\n",
      "    values: 0.877920064666115\n",
      "    values: 0.658283652355672\n",
      "    values: 0.36857864756244674\n",
      "    values: 0.530157555200853\n",
      "    values: 0.34183838767424424\n",
      "    values: 0.2729533988639926\n",
      "    values: 0.8156090783031779\n",
      "    values: 0.2638208836938779\n",
      "    values: 0.48741219981945405\n",
      "    values: 0.27396550303338907\n",
      "    values: 0.3570731065760536\n",
      "    values: 0.6106294928408991\n",
      "    values: 0.48521455473900355\n",
      "    values: 0.27636567274268\n",
      "    values: 0.5115665470448926\n",
      "    values: 0.4296451756950952\n",
      "    values: 0.07822243419838792\n",
      "    values: 0.5129113065705019\n",
      "    values: 0.5275964697175866\n",
      "    values: 0.8680526618713468\n",
      "    values: 0.9931993779512479\n",
      "    values: 0.5700505374808907\n",
      "    values: 0.18648636809161856\n",
      "    values: 0.8126782876701942\n",
      "    values: 0.5757764515674432\n",
      "    values: 0.9812369649104862\n",
      "    values: 0.1767305130616016\n",
      "    values: 0.2332112431783091\n",
      "    values: 0.37171481763124914\n",
      "    values: 0.6604781534451094\n",
      "    values: 0.7288104296673006\n",
      "    values: 0.08867812462148938\n",
      "    values: 0.7113488119471519\n",
      "    values: 0.6076531448323415\n",
      "    values: 0.48151791957579915\n",
      "    values: 0.7657466199156793\n",
      "    values: 0.7514395999437494\n",
      "    values: 0.7196029464473742\n",
      "    values: 0.876174802090644\n",
      "    values: 0.3491125681069417\n",
      "    values: 0.7093588345472498\n",
      "    values: 0.7555475008164038\n",
      "    values: 0.563492855525081\n",
      "    values: 0.1825410644218276\n",
      "    values: 0.9578608832195183\n",
      "    values: 0.9429716868236594\n",
      "    values: 0.7567589659306391\n",
      "    values: 0.5824410312188966\n",
      "    values: 0.39987288906208784\n",
      "    values: 0.6871346148063908\n",
      "    values: 0.07958339716779195\n",
      "    values: 0.7488788906174467\n",
      "    values: 0.4504334694990618\n",
      "    values: 0.937154000460234\n",
      "    values: 0.6161862682408318\n",
      "    values: 0.14232391934240418\n",
      "    values: 0.7051409704133775\n",
      "    values: 0.554246978117284\n",
      "    values: 0.5409135294512057\n",
      "    values: 0.6232907292193071\n",
      "    values: 0.7924326899453114\n",
      "    values: 0.07900784626277102\n",
      "    values: 0.7311328133696953\n",
      "    values: 0.10879879872611475\n",
      "    values: 0.6883285801280582\n",
      "    values: 0.4471349275407557\n",
      "    values: 0.04263383220408479\n",
      "    values: 0.048483081893327395\n",
      "    values: 0.9068155019549612\n",
      "    values: 0.04018171506169255\n",
      "    values: 0.3107078521213035\n",
      "    values: 0.6881708298352383\n",
      "    values: 0.9311764679633986\n",
      "    values: 0.10148788622035942\n",
      "    values: 0.8577863329184886\n",
      "    values: 0.2819715083557457\n",
      "    values: 0.8543442999842149\n",
      "    values: 0.5094937033682979\n",
      "    values: 0.7867208110859827\n",
      "    values: 0.07775847041676798\n",
      "    values: 0.3157778964700201\n",
      "    values: 0.6575468146276017\n",
      "    values: 0.713653422836335\n",
      "    values: 0.693893431275562\n",
      "    values: 0.6888791061622257\n",
      "    values: 0.9936624300495328\n",
      "    values: 0.15402012437652002\n",
      "    values: 0.06845411988012884\n",
      "    values: 0.04292562769682795\n",
      "    values: 0.28978983828939153\n",
      "    values: 0.04054542197846511\n",
      "    values: 0.885154054310623\n",
      "    values: 0.11600231965213603\n",
      "    values: 0.6057493825981799\n",
      "    values: 0.38917891830081686\n",
      "    values: 0.9167964900708111\n",
      "    values: 0.1414291928923963\n",
      "    values: 0.046840221209781374\n",
      "    values: 0.7879633178240502\n",
      "    values: 0.9327632079422756\n",
      "    values: 0.9740906447697041\n",
      "    values: 0.3753163440049284\n",
      "    values: 0.736786611530133\n",
      "    values: 0.7519448068825569\n",
      "    values: 0.5348406338310284\n",
      "    values: 0.6097736767799726\n",
      "    values: 0.6707171921612622\n",
      "    values: 0.4850489360560779\n",
      "    values: 0.06435899958314162\n",
      "    values: 0.4069752865709426\n",
      "    values: 0.11677526900516966\n",
      "    values: 0.25485384753477425\n",
      "    values: 0.1042817017080313\n",
      "    values: 0.23546499511754904\n",
      "    values: 0.6477076533486364\n",
      "    values: 0.0969132590922418\n",
      "    values: 0.18063296631981052\n",
      "    values: 0.19125842188089703\n",
      "    values: 0.9398630903350261\n",
      "    values: 0.04917178111818554\n",
      "    values: 0.07001026561293933\n",
      "    values: 0.7851696552390228\n",
      "    values: 0.2519985786336376\n",
      "    values: 0.24578980986087484\n",
      "    values: 0.8785369343912791\n",
      "    values: 0.06851557237775674\n",
      "    values: 0.0791052985909998\n",
      "    values: 0.7607754240657496\n",
      "    values: 0.3043556732133149\n",
      "    values: 0.1412923208586616\n",
      "    values: 0.07861697162268466\n",
      "    values: 0.5855683622249785\n",
      "    values: 0.015474296364465734\n",
      "    values: 0.3626273627096026\n",
      "    values: 0.6142006101072658\n",
      "    values: 0.15317882949135564\n",
      "    values: 0.4760634056296531\n",
      "    values: 0.1281058951147056\n",
      "    values: 0.9817978623276874\n",
      "    values: 0.2822779830057438\n",
      "    values: 0.18899912943255026\n",
      "    values: 0.15040460340302597\n",
      "    values: 0.47323549071920257\n",
      "    values: 0.39294851264160224\n",
      "    values: 0.5607007328691221\n",
      "    values: 0.8886912337489032\n",
      "    values: 0.34556280687170704\n",
      "    values: 0.1995031054004417\n",
      "    values: 0.19023269258849673\n",
      "    values: 0.7983547694883882\n",
      "    values: 0.5753693096868119\n",
      "    values: 0.07589245456486815\n",
      "    values: 0.9088265096402942\n",
      "    values: 0.06549385727779888\n",
      "    values: 0.9160717605188328\n",
      "    values: 0.7150161057039807\n",
      "    values: 0.9952194736289688\n",
      "    values: 0.2609461687442499\n",
      "    values: 0.8092499916174705\n",
      "    values: 0.08873534152440599\n",
      "    values: 0.14229174863771732\n",
      "    values: 0.9667719130844892\n",
      "    values: 0.650434896401207\n",
      "    values: 0.7464119737663227\n",
      "    values: 0.9302255414481282\n",
      "    values: 0.07560836125116543\n",
      "    values: 0.35188095761446636\n",
      "    values: 0.4346279847365707\n",
      "    values: 0.15909349868445533\n",
      "    values: 0.3240893664304797\n",
      "    values: 0.6476345237575679\n",
      "    values: 0.9706901495418921\n",
      "    values: 0.9313898241008745\n",
      "    values: 0.735725791605063\n",
      "    values: 0.5024632164435275\n",
      "    values: 0.4422749063180411\n",
      "    values: 0.7570735718033837\n",
      "    values: 0.7043073835290968\n",
      "    values: 0.8309032045222933\n",
      "    values: 0.9306319143793531\n",
      "    values: 0.018737570455761965\n",
      "    values: 0.29705523369946696\n",
      "    values: 0.9148901050717295\n",
      "    values: 0.4252276286471166\n",
      "    values: 0.0698628099684322\n",
      "    values: 0.12942926304415958\n",
      "    values: 0.9895572467028818\n",
      "    values: 0.5612464392047\n",
      "    values: 0.9622499724232425\n",
      "    values: 0.2957794985780192\n",
      "    values: 0.3123704593143455\n",
      "    values: 0.521813736272192\n",
      "    values: 0.15734837674333113\n",
      "    values: 0.9300686063703849\n",
      "    values: 0.025894431641466098\n",
      "    values: 0.19989143558250877\n",
      "    values: 0.8098083114545819\n",
      "    values: 0.45823886699942595\n",
      "    values: 0.6444181383359471\n",
      "    values: 0.39598416215941235\n",
      "    values: 0.8287630113507214\n",
      "    values: 0.03847956709186062\n",
      "    values: 0.5920790051769474\n",
      "    values: 0.7060219874872269\n",
      "    values: 0.57182345133701\n",
      "    values: 0.34681824431555863\n",
      "    values: 0.772127135080517\n",
      "    values: 0.6981691534033239\n",
      "    values: 0.09376915332560876\n",
      "    values: 0.02413035563680399\n",
      "    values: 0.8430329122570326\n",
      "    values: 0.314129807956945\n",
      "    values: 0.8646694892839248\n",
      "    values: 0.8732961980982168\n",
      "    values: 0.060043785004324834\n",
      "    values: 0.8087740816427692\n",
      "    values: 0.44896486902443433\n",
      "    values: 0.22375067593049192\n",
      "    values: 0.3579990468342229\n",
      "    values: 0.6256172132546013\n",
      "    values: 0.744831821030211\n",
      "    values: 0.09913673838044523\n",
      "    values: 0.37043901594583506\n",
      "    values: 0.23965358822099825\n",
      "    values: 0.1693343930184279\n",
      "    values: 0.8899237831783093\n",
      "    values: 0.2696736277318531\n",
      "    values: 0.6409735451548131\n",
      "    values: 0.008784220443631363\n",
      "    values: 0.9010537138158498\n",
      "    values: 0.2730262930932933\n",
      "    values: 0.005027750666353303\n",
      "    values: 0.31242724057477833\n",
      "    values: 0.2992320102738386\n",
      "    values: 0.1808866764459649\n",
      "    values: 0.5015938018724175\n",
      "    values: 0.8752247333851866\n",
      "    values: 0.671587983854636\n",
      "    values: 0.9688107094933738\n",
      "    values: 0.2703009118847447\n",
      "    values: 0.7728503199572744\n",
      "    values: 0.03368798626783731\n",
      "    values: 0.6600590631797197\n",
      "    values: 0.12051474241524607\n",
      "    values: 0.7896141448339796\n",
      "    values: 0.33202267059904\n",
      "    values: 0.5557459223563734\n",
      "    values: 0.8896362802777633\n",
      "    values: 0.006455206574829897\n",
      "    values: 0.36775163800881294\n",
      "    values: 0.5710844186942228\n",
      "    values: 0.24599089823130738\n",
      "    values: 0.6455246922751214\n",
      "    values: 0.9629833699840081\n",
      "    values: 0.8906004020972904\n",
      "    values: 0.11043493825427708\n",
      "    values: 0.07817133215027727\n",
      "    values: 0.14883312839207086\n",
      "    values: 0.18896407276173832\n",
      "    values: 0.006405622112598097\n",
      "    values: 0.18174074752718572\n",
      "    values: 0.7084031453965034\n",
      "    values: 0.6606144469797846\n",
      "    values: 0.5871042427566378\n",
      "    values: 0.43861253376889464\n",
      "    values: 0.12669077553771302\n",
      "    values: 0.19683468767832835\n",
      "    values: 0.04970640434258533\n",
      "    values: 0.5726524841381484\n",
      "    values: 0.981169806297863\n",
      "    values: 0.3209577095574365\n",
      "    values: 0.4846256682190856\n",
      "    values: 0.16997694913532146\n",
      "    values: 0.16868811200058542\n",
      "    values: 0.8963932611562712\n",
      "    values: 0.2512771807214328\n",
      "    values: 0.4969903234144778\n",
      "    values: 0.5073810961301962\n",
      "    values: 0.7048461763567072\n",
      "    values: 0.45230294590520503\n",
      "    values: 0.777021844693988\n",
      "    values: 0.800977925138967\n",
      "    values: 0.016247130659590825\n",
      "    values: 0.6313971018877517\n",
      "    values: 0.5624121894706194\n",
      "    values: 0.0239147231347\n",
      "    values: 0.9631320285887279\n",
      "    values: 0.06418590190826412\n",
      "    values: 0.05248658465196909\n",
      "    values: 0.845250343059501\n",
      "    values: 0.637831910340642\n",
      "    values: 0.33922261657075126\n",
      "    values: 0.6552333913452396\n",
      "    values: 0.7051885178865996\n",
      "    values: 0.7681770568758165\n",
      "    values: 0.7989325307716076\n",
      "    values: 0.019107745569871937\n",
      "    values: 0.7946235532817478\n",
      "    values: 0.5005678164050742\n",
      "    values: 0.015315627382751495\n",
      "    values: 0.8834804848065625\n",
      "    values: 0.7500302014792979\n",
      "    values: 0.46004316564622183\n",
      "    values: 0.0731941569888912\n",
      "    values: 0.03214823737700068\n",
      "    values: 0.2999932019225978\n",
      "    values: 0.972776229543541\n",
      "    values: 0.03710282505599105\n",
      "    values: 0.11482962960476517\n",
      "    values: 0.2482980992380871\n",
      "    values: 0.3102809224558386\n",
      "    values: 0.3912102030029877\n",
      "    values: 0.3262647338659558\n",
      "    values: 0.5161124532548613\n",
      "    values: 0.11872389458119992\n",
      "    values: 0.5953730774499911\n",
      "    values: 0.15000444444064087\n",
      "    values: 0.20097883771091363\n",
      "    values: 0.4495728961691686\n",
      "    values: 0.5191506168846992\n",
      "    values: 0.36064870234181834\n",
      "    values: 0.542684393859647\n",
      "    values: 0.6274453466242965\n",
      "    values: 0.6145302438770212\n",
      "    values: 0.8711863790525174\n",
      "    values: 0.2351843798203309\n",
      "    values: 0.9113434300193178\n",
      "    values: 0.06089294763325537\n",
      "    values: 0.8203517960556094\n",
      "    values: 0.9180327484300594\n",
      "    values: 0.34905965824462926\n",
      "    values: 0.6187869948765344\n",
      "    values: 0.5623947125228388\n",
      "    values: 0.14231915340169954\n",
      "    values: 0.1088444727013872\n",
      "    values: 0.7721060893741788\n",
      "    values: 0.2600628968961195\n",
      "    values: 0.14196593387017487\n",
      "    values: 0.6201059966784203\n",
      "    values: 0.7592180251591637\n",
      "    values: 0.25283286048847164\n",
      "    values: 0.9978210604275419\n",
      "    values: 0.7380790999425393\n",
      "    values: 0.7485575725821964\n",
      "    values: 0.08512223751242987\n",
      "    values: 0.012579629860025587\n",
      "    values: 0.4781944384297896\n",
      "    values: 0.7647727693833505\n",
      "    values: 0.03310836060181288\n",
      "    values: 0.30474966902472633\n",
      "    values: 0.5813831715028552\n",
      "    values: 0.15561070625440676\n",
      "    values: 0.31580173219869523\n",
      "    values: 0.14144304394362517\n",
      "    values: 0.6822783309708192\n",
      "    values: 0.7745950463535098\n",
      "    values: 0.11329166397284163\n",
      "    values: 0.3765425112990788\n",
      "    values: 0.7756608646870186\n",
      "    values: 0.026401166859190583\n",
      "    values: 0.7086109599000308\n",
      "    values: 0.6683704537092324\n",
      "    values: 0.898865629534389\n",
      "    values: 0.5614105941976864\n",
      "    values: 0.9295770629451052\n",
      "    values: 0.08457971242188356\n",
      "    values: 0.09731557673982782\n",
      "    values: 0.717528461447746\n",
      "    values: 0.4435589819450947\n",
      "    values: 0.09267859954832602\n",
      "    values: 0.22075050237890836\n",
      "    values: 0.5440385992306206\n",
      "    values: 0.28869038035142713\n",
      "    values: 0.6414667331700948\n",
      "    values: 0.1032554273547387\n",
      "    values: 0.5624248200788803\n",
      "    values: 0.7529239720921993\n",
      "    values: 0.37174394855131276\n",
      "    values: 0.1874762518156724\n",
      "    values: 0.8450512847434836\n",
      "    values: 0.4677247624980835\n",
      "    values: 0.4925384698119981\n",
      "    values: 0.3922452617036699\n",
      "    values: 0.5754634838509465\n",
      "    values: 0.8978127732090391\n",
      "    values: 0.6541189224722362\n",
      "    values: 0.8829980807519465\n",
      "    values: 0.4164403725342146\n",
      "    values: 0.27701332777131693\n",
      "    values: 0.7442734060764078\n",
      "    values: 0.43709317554404625\n",
      "    values: 0.11069489900105622\n",
      "    values: 0.04372423154286542\n",
      "    values: 0.6188816320735743\n",
      "    values: 0.5226056312182997\n",
      "    values: 0.07721293584340905\n",
      "    values: 0.447249184479529\n",
      "    values: 0.7483750281727048\n",
      "    values: 0.9543491190476169\n",
      "    values: 0.03283751989472039\n",
      "    values: 0.23753708777980254\n",
      "    values: 0.597696738841465\n",
      "    values: 0.07290028826935813\n",
      "    values: 0.752427774864024\n",
      "    values: 0.22213480132387098\n",
      "    values: 0.7981321800944998\n",
      "    values: 0.6992784626450497\n",
      "    values: 0.6506205013874153\n",
      "    values: 0.6110762048950947\n",
      "    values: 0.7411405767899414\n",
      "    values: 0.32300169510968435\n",
      "    values: 0.3249173253006642\n",
      "    values: 0.19765556725943279\n",
      "    values: 0.907607727419948\n",
      "    values: 0.049159598848120045\n",
      "    values: 0.11813253996195405\n",
      "    values: 0.5352296928708075\n",
      "    values: 0.2894959603816454\n",
      "    values: 0.41085713895845977\n",
      "    values: 0.16505693962086831\n",
      "    values: 0.9327669948182279\n",
      "    values: 0.18723316817419433\n",
      "    values: 0.6897462433648315\n",
      "    values: 0.5541311610792786\n",
      "    values: 0.3631240951438194\n",
      "    values: 0.7819798670964472\n",
      "    values: 0.8567068382704525\n",
      "    values: 0.13741698440528027\n",
      "    values: 0.8841391969091527\n",
      "    values: 0.20271236056898578\n",
      "    values: 0.09242464895078384\n",
      "    values: 0.9766739478136849\n",
      "    values: 0.7350808985634926\n",
      "    values: 0.7888170664033589\n",
      "    values: 0.7870899672144818\n",
      "    values: 0.9998261360560717\n",
      "    values: 0.8243184647258451\n",
      "    values: 0.8126889766066961\n",
      "    values: 0.8919538365102384\n",
      "    values: 0.6908472905653973\n",
      "    values: 0.8348379895761122\n",
      "    values: 0.9315872393965912\n",
      "    values: 0.19240208610841958\n",
      "    values: 0.2917105898928204\n",
      "    values: 0.9970238972236198\n",
      "    values: 0.5285998562610984\n",
      "    values: 0.9651870370929523\n",
      "    values: 0.4773414208202188\n",
      "    values: 0.7080265528775707\n",
      "    values: 0.5531478401143465\n",
      "    values: 0.20067402465106143\n",
      "    values: 0.0910473652431597\n",
      "    values: 0.5566812653679821\n",
      "    values: 0.1895863277679234\n",
      "    values: 0.18972017452285228\n",
      "    values: 0.21573093078266958\n",
      "    values: 0.28819941962670015\n",
      "    values: 0.7554714567345686\n",
      "    values: 0.3299833119023001\n",
      "    values: 0.4823634605797319\n",
      "    values: 0.7347895854119262\n",
      "    values: 0.5419590120717362\n",
      "    values: 0.5796473212803687\n",
      "    values: 0.15895666421627253\n",
      "    values: 0.27820651324314327\n",
      "    values: 0.5307501316553745\n",
      "    values: 0.6017282898064901\n",
      "    values: 0.652350809017883\n",
      "    values: 0.5866517445716298\n",
      "    values: 0.33780409364726616\n",
      "    values: 0.6134773116492235\n",
      "    values: 0.9635631255069724\n",
      "    values: 0.3155534987404389\n",
      "    values: 0.8445536335795542\n",
      "    values: 0.5472537300640109\n",
      "    values: 0.730739888926089\n",
      "    values: 0.9379887639741543\n",
      "    values: 0.647185622812615\n",
      "    values: 0.9525324000917615\n",
      "    values: 0.12124162825886475\n",
      "    values: 0.9820500980791068\n",
      "    values: 0.6351761819009761\n",
      "    values: 0.5096275492626993\n",
      "    values: 0.7828497538276272\n",
      "    values: 0.15893915471373588\n",
      "    values: 0.3451990159398336\n",
      "    values: 0.6475856433679662\n",
      "    values: 0.36241675959479336\n",
      "    values: 0.05223772741291266\n",
      "    values: 0.4426143815083019\n",
      "    values: 0.7483366979450781\n",
      "    values: 0.15732287826242441\n",
      "    values: 0.5505937863592139\n",
      "    values: 0.5838898870400506\n",
      "    values: 0.8015764619260546\n",
      "    values: 0.36608357653743173\n",
      "    values: 0.7363443956244988\n",
      "    values: 0.0735287974062151\n",
      "    values: 0.47041809681173274\n",
      "    values: 0.21865426368912189\n",
      "    values: 0.4426815449770912\n",
      "    values: 0.057378160647420295\n",
      "    values: 0.7586390554209108\n",
      "    values: 0.26421578613496544\n",
      "    values: 0.8819407711539675\n",
      "    values: 0.5914772064287014\n",
      "    values: 0.8567896103014099\n",
      "    values: 0.8450385566992789\n",
      "    values: 0.24089876508232289\n",
      "    values: 0.9161708472348465\n",
      "    values: 0.0018941740885223268\n",
      "    values: 0.6282595126309252\n",
      "    values: 0.29861601609411925\n",
      "    values: 0.5601121724735366\n",
      "    values: 0.6387182386062428\n",
      "    values: 0.8137147846879669\n",
      "    values: 0.9337864597692335\n",
      "    values: 0.8414109922079155\n",
      "    values: 0.982870606046976\n",
      "    values: 0.9927274415841693\n",
      "    values: 0.8591044305852436\n",
      "    values: 0.10660635287001141\n",
      "    values: 0.6986013233859512\n",
      "    values: 0.025221674231743152\n",
      "    values: 0.8881888704557174\n",
      "    values: 0.5108908540467048\n",
      "    values: 0.07870239201369944\n",
      "    values: 0.26108951344586095\n",
      "    values: 0.6012072043277703\n",
      "    values: 0.6715219596531513\n",
      "    values: 0.5119352594863543\n",
      "    values: 0.7155813301808434\n",
      "    values: 0.6443101319727587\n",
      "    values: 0.8372477307086283\n",
      "    values: 0.4821563386787143\n",
      "    values: 0.7355682681078428\n",
      "    values: 0.8354490487874056\n",
      "    values: 0.35903954925805637\n",
      "    values: 0.8342739311609206\n",
      "    values: 0.17795820834568554\n",
      "    values: 0.7876698854014467\n",
      "    values: 0.13293124638987963\n",
      "    values: 0.2618021890579171\n",
      "    values: 0.8442123309423877\n",
      "    values: 0.255222477754938\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "meta {\n",
      "  puid: \"ubj03lvd5hhkivjf4isd5nhfh6\"\n",
      "  requestPath {\n",
      "    key: \"mnist-model\"\n",
      "    value: \"seldonio/tfserving-proxy_rest:0.6\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"t:0\"\n",
      "  names: \"t:1\"\n",
      "  names: \"t:2\"\n",
      "  names: \"t:3\"\n",
      "  names: \"t:4\"\n",
      "  names: \"t:5\"\n",
      "  names: \"t:6\"\n",
      "  names: \"t:7\"\n",
      "  names: \"t:8\"\n",
      "  names: \"t:9\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 10\n",
      "    values: 3.32885639e-23\n",
      "    values: 1.21791702e-33\n",
      "    values: 0.113864869\n",
      "    values: 0.886001587\n",
      "    values: 1.36405575e-31\n",
      "    values: 9.80257901e-05\n",
      "    values: 1.05495472e-14\n",
      "    values: 8.12247104e-16\n",
      "    values: 3.54823496e-05\n",
      "    values: 1.82557789e-18\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,784))\n",
    "print(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"tfserving\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/tfserving/samples/mnist_rest.yaml"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Serve MLFlow Elasticnet Wines Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34;01mapiVersion\u001b[39;49;00m: machinelearning.seldon.io/v1alpha2\r\n",
      "\u001b[34;01mkind\u001b[39;49;00m: SeldonDeployment\r\n",
      "\u001b[34;01mmetadata\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: mlflow\r\n",
      "\u001b[34;01mspec\u001b[39;49;00m:\r\n",
      "  \u001b[34;01mname\u001b[39;49;00m: wines\r\n",
      "  \u001b[34;01mpredictors\u001b[39;49;00m:\r\n",
      "  - \u001b[34;01mgraph\u001b[39;49;00m:\r\n",
      "      \u001b[34;01mchildren\u001b[39;49;00m: []\r\n",
      "      \u001b[34;01mimplementation\u001b[39;49;00m: MLFLOW_SERVER\r\n",
      "      \u001b[34;01mmodelUri\u001b[39;49;00m: gs://seldon-models/mlflow/elasticnet_wine\r\n",
      "      \u001b[34;01mname\u001b[39;49;00m: classifier\r\n",
      "    \u001b[34;01mname\u001b[39;49;00m: default\r\n",
      "    \u001b[34;01mreplicas\u001b[39;49;00m: 1\r\n"
     ]
    }
   ],
   "source": [
    "!pygmentize ../servers/mlflowserver/samples/elasticnet_wine.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io/mlflow created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f ../servers/mlflowserver/samples/elasticnet_wine.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "deployment \"wines-default-8c791aa\" successfully rolled out\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl rollout status deploy/wines-default-057e362"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "from seldon_core.seldon_client import SeldonClient\n",
    "sc = SeldonClient(deployment_name=\"mlflow\",namespace=\"seldon\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Success:True message:\n",
      "Request:\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 11\n",
      "    values: 0.9370144045923416\n",
      "    values: 0.8142853007643482\n",
      "    values: 0.19471732807241948\n",
      "    values: 0.5155919467447084\n",
      "    values: 0.31446447854903803\n",
      "    values: 0.6282233088869803\n",
      "    values: 0.7504781874678473\n",
      "    values: 0.6568746664582721\n",
      "    values: 0.32870680648593553\n",
      "    values: 0.09666979723379854\n",
      "    values: 0.64486945913671\n",
      "  }\n",
      "}\n",
      "\n",
      "Response:\n",
      "meta {\n",
      "  puid: \"j2qdunn3uebeq5oj7gunmpr81u\"\n",
      "  requestPath {\n",
      "    key: \"classifier\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    values: 5.231826975896663\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "r = sc.predict(gateway=\"ambassador\",transport=\"rest\",shape=(1,11))\n",
    "print(r)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment.machinelearning.seldon.io \"mlflow\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f ../servers/mlflowserver/samples/elasticnet_wine.yaml"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
